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Abstract 

Evolutionary game dynamics in finite populations provides a new framework to understand the selection of traits with 
frequency-dependent fitness. Recently, a simple but fundamental law of evolutionary dynamics, which we call cr law, 
describes how to determine the selection between two competing strategies: in most evolutionary processes with two 
strategies, A and B, strategy A is favored over B in weak selection if and only if crR + S > T + crP. This relationship 
holds for a wide variety of structured populations with mutation rate and weak selection under certain assumptions. 
In this paper, we propose a model of games based on a community-structured population and revisit this law under the 
Moran process. By calculating the average payoffs of A and B individuals with the method of effective sojourn time, 
we find that cr features not only the structured population characteristics but also the reaction rate between individuals. 
That's to say, an interaction between two individuals are not uniform, and we can take cr as a reaction rate between any 
two individuals with the same strategy. We verify this viewpoint by the modified replicator equation with non-uniform 
interaction rates in a simplified version of the prisoner's dilemma game (PDG). 

Keywords: Evolutionary game theory, Sojourn time, Non-uniform interaction rate, Simplified PDG. 



1. Introduction 



E volutionary game theory was originally introduced as a tool for studying animal behavior ( Mavnard and Price! 



1973 



Mavnardill982h but has bec ome a general approach that transcends almost every aspect of evolutionary biol- 



ogy ( Nowak and Sigmundi 2004a), and provides a framework to unde r stand the dynamics of frequency-dependen t 
selection dHofbauer and Sigmundi 1 1998t bintisl l2000t ICressmanl 120031: iNowak and Sigmundi l2004al; Nowaki l2006t 



Antal et all l2009at IWu et allbOloilGokhale and TraulsenibOlTtlTarnita et all 1201 lb 

The traditional appr oach of evolutionary game theory us es deterministic dynamics to describe infinitely large, 
well-mixed populations (Hofbauer and Sigmundi [l988[|2003h To understand evolutionary game dynamics in finite- 
sized populations, a stochastic approach i s deve loped ( Schaffei , 1988 : Fogel et al. . 1998 : Ficici et al. . 2000l : Alos-Ferrer 
2003t iPerd 120071 iTraulsen and Nowaki 120071) . A crucial quantity in the stochastic approach i s the fixation prob- 
ability of strategies which relies on the individual's reproduction ability ( Nowak et al. , 2004bl Taylor et al. , 2004 ; 
Imhof and Nowak, 12004 Nowak, 12004 Traulsen et all l2006at iLessard and Ladretl 120071: iBomze an d Pawl owitschl 
20081) . In the limit of weak selection, for a neutral mutant, probability p^, that an individual using strategy A will 
invade and take over the whole population of individuals using strategy B, is equal to the reciprocal of the population 
size, i.e., 1 /N. Therefore, the selection favors the fixation of invading strategy A if p& > l/N under the weak selection. 

Evolutionary game dynamics are also affected by the structure of population. Lots of early works focused on 
spatial games with regular lattices (Nowak and May, 1992; Nakamaru et al., 1997; Hauert and Doebeli, 2004; Szabo 
and Fat h, 1998; Szabo et al., 2000), which recently, have been expanded to general structured populations, such as 



ana rat n, iwa; azabo et al., ZUUU), wnicn re cently, nave been expanded to gen eral structured populatio ns, sucn a 
graphs dLieberman et all 120051 lOhtsuki et al.ll2006allOhtsuki and Nowaki |2006bl:lohtsuki et alll2007allTavlor et al 



2007l:ICao and Lill2008llFu et all 120091: iLi and Caoll2009F 



Perc and Wan 



2010l: IWang etlU 1201 lh . phenotvpe space 



dAntal et all l2009bl)~ and set structured populations (ITarnita et all l2009alh In a word, a population structure specifies 
who interacts with whom, and it greatly affects the outcome of an evolution. If the fitn ess of an individual is de termined 
by its interactions with others, then we are in the world of evolutionary game theory dNathanson et all l2009h . 
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Consider a game of two strategies, A and B, in a population of fixed size N. Mutual cooperation is rewarded by R 
for each player, whereas mutual defection pays each player a punishment P. If a cooperator plays against a defector, 
the former gets the suckers payoff S and the latter gets the temptation to defect T. Thus, the interactions are given by 
the payoff matrix 

A B 

A R S (1) 
B T P 

Each individual obtains a payoff by interacting with others according to the population structure, and reproduces 
either genetically or culturally with a rate proportional to its payoff. Of course, reproduction is subject to mutation. 
Whenever an individual reproduces, the offspring adopts the parent's strategy with probability 1 - v and a random 
strategy with probability v. To make the evoluti onary dynamics of the process well approximated by an embedded 
Markov chain on the pure states (IWu et all 1201 lb . it's considered that mutation rate v < (NlnN)~ l <k 1. 



It is said that strategy A is selected over stra tegy B if it is more abundant in the stationary distribution of the 
mutation-selection process. iTarnita et al.1 (12009bl) showed that with the weak selection, the condition that strategy A 
is more abundant than strategy B in the stationary distribution of the mutation-selection process is 



o-R + S >T + o-P 



(2) 



From this inequality, we know that the condition specifying which strategy is more abundant is a linear inequality 
of the payoff values, R, S , T, P. It's pointed o ut that parameter cr, reflects how the structured populations influence 
the evolutionary dynamics ( Nowak et al. . 20 id) , and this inequality holds for a wide variety of population structures , 



including well mixed populations, graphs, phenotype space and set structured populations (INathanson et al 
Conveniently, we call this simple but fundamental law as the cr law. 



2009) 



R ecently, attention are focused on t he population structures allowing selection at multiple leve ls (Girvan and N ewman , 

2002[ iNewmanl 120061: iTraulsen et all l2005al l2006bl l2008t IWang et all 1201 it lHaueit and Imhofl 1201 it IWang et all 
2012b . In this case, a population is divided into groups (or communities), and the selection between individuals can 
be either in a community or among several communities. Following this line, we propose a model of games on a 
community-structured population based on the graph theory with a finite size. We verify that evolutionary games on 
the community-structured population obeys the cr law from the viewpoint of interaction under the Moran proces s with 
sojourn time. Along the path of an invasion attempt, we find that the probability of an interaction between any two 
individuals with the same strategy is cr times of that between different strategies. In other words, cr represents the 
reaction rate between two individuals with the same strategy, and quantifies the degree that individuals using the same 
strategy are more (cr > 1) or less (cr < 1) likely to interact than individuals using different strategies. 



2. cr law in community-structured populations 

2.1. Model of community structures 

Consider a game played in a finite population of the fixed size H distributed over M communities (each community 
has the same size N, and any two of which have none common member) with strategies A and B. Each individual 
obtains a payoff according to the payoff matrix (1). At each time step, a single individual is selected proportional 
to its fitness for reproduction, and the offspring either replaces a randomly chosen indivi dual in this community o r 



moves to another community at any time step to replaces a randomly chosen individual. In lHauert and Imhofl (1201 II) . 
the authors discussed the evolutionary dynamics in a deme structured population, and assumed that the migration 
rate is proportional to deme's fitness. In this paper, we suppose that there exists a graph in the structured population 
connecting communities, and the migration occurs between two communities if there is an edge between them (See 
Fig. 1). 

Label all communities in the structured populations with / = 1, 2, • • • , M. The probability that a selected individual 
belongs to community I migrates to community m is given by Ai m . Hence the migration process is determined by an 
M xM matrix, A = [Ai,„], An is the probability that an individual stays in community /. Note that An + Yjmen, ^im = 1, 
where Q/ denotes the set of all the communities that individuals in community I can migrate to. Thus A is a st ochastic 



matrix with an eigenvalue 1, and no eigenvalue has an absolute value greater than 1 (Godsil and Rovle, 2001 ) 



2 



Assume that a mutant A-individual (individual using strategy A) is produced from a community / with pure strategy 
B. This mutant replaces a randomly chosen individual in community / with probability An, or migrates to community 
m and replaces a randomly chosen individual in community m with probability Ai m , m e £2/. Since the graph connects 
all the communities as its nodes, there exists a path along which a mutant beginning in community 1,1 € {1,2, - ■ ■ , M\ 
can be fixed in the whole population (note that the path is not unique). An example that the fixation process of a 
single mutant in the whole community-structured population is shown in Fig. 2, where the fixation of a mutant from 
community 1 to take over the whole population corresponds to a path. 

2.2. cr law in community-structured population with sojourn time 

Assume that a migration occurs very rarely and much less than mutation (Ai„, <k v < {NlnNY 1 <k \,m e Q;). 
In the limit of A\ m — > and v — > 0, the population spends almost all the time in the homogeneous states with all 
individuals of type A or type B. Since mutations occur much more frequently than mig rations, a migrant has either 



invaded the whole community or been eliminated before the next migration takes place (ITraulsen et al.l 1200 6b). Due 
to this special structure, the interactions within a community is more frequent than those between communities. This 
allows us to use the meth od of sojourn t ime in the Moran process which was applied to explain the one-third law of 



evolutionary dynamics in lOhtsuki et al.l (12007bl) . 

Suppose that a single individual from /-community is selected proportional to its fitness for reproduction. This 
newborn has to stay or migrate to another community and replaces a randomly selected individual, and the total 
number of the population keeps unchanged. Based on the migration matrix A constructed previously, we know that 
the newborn stays in community / with probability An, and migrate to community m with probability A/ m , where 

An » Aim, m + I- 

First, we discuss the Moran process that happens in /-community with probability A//. Among the total /V individ- 
uals in community /, j of them follow strategy A, and N - j follow strategy B. Introduce a parameter a> to measure 
the intensity of selection. The corresponding fitness of A and B individuals are given by 

fj — I — to + uFj 
gj — 1 — co + a)Gj 

where Fj and Gj are the expected payoffs for A and B, respectively. In this paper, we consider the weak selection, i.e., 
a> — > 0. In the state space {0, • ■ • , /V}, there are two absorbing states, 7 = (which means A has become extinct) and 
j — N (which means A has reached fixation). Then the transition probability Pjj±\ is given by 

PjJ+l =(N - ,/V.Y • jfj/[jfj + (N- j) gj ] 

Pjj-i = j/N ■ |(.V - j) gj ] I [jfj + (.V- j) 8j ] (3) 
Pi,i = 1 -Pi,-: - Pi, : 

Consider a path along which a mutant invades a resident population from state j (1 < j < N — 1). Since the mutant 
eventually g ets absorbed into eithe r state or state /V, we are interested in how much time is spent at state j along 
the path. In lOhtsuki et al. I d2007bl) . the authors called it the sojourn time. Given all paths that start at state i, let fy 



be the m ean sojourn tim e at state j before the absorption into either state or state N, which is described as (see, for 
example, Ewensl ( 2004 )). 

k' 

t,j = 2^ Pfkhj + 5,j, toj = t N j = (4) 

k=0 

In particular, if we start from a single mutant with no loss of generality, i.e., i = 1, then Eq. (4) can be simplified as 

~hj = N/j. (5) 

This means that the stochastic process spends most of the time around the absorbing state, j = 0. Besides, the mean 
effective sojourn time Ty at state j for paths that start at state i is given by Ty = (p jj+i + Pjj-\)tjj. Now, we only 
concern the mean effective sojourn time f\j which starts from a single mutant under the neutral drift. Since the fitness 
fj and gj are very close to 1 under weak selection, with Eqs. (3) and (5), we obtain 

7 U = (PjJ+i +Pj,i-i)~j 

= K.V - j)/N ■ jfj/ljfj + (N- j) gj ] + j/N ■ |(.V - j)gj]/[jfj + (N- j)gj])N/j (6) 
« 2(N - j)/N 
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We now can count the average number of games played by A -individual and /J-individual along a path of invasion. 
At state j, an A-individual meets (j - 1) A-individuals and (N - j) /J-individuals, while a /J-individual meets j A- 
individuals and (N — j — 1) /j -individuals. The aver age number of games played by A-individual and /J-individual at 
state j can be summarized as (lOhtsuki et al. ■ l2007bh 



A^A A —> B \ _l j -I N-j 
B -> A B^B J. \ j N-j-l ' 



Therefore, the effective number of encounters are 

N-l 

;=i 



7-1 N-j 
j N-j-l 



27=7 £0' - - J) Z£T W - mn - j) 

%M - A S^Ti 1 M N - j - D(N - j) 

z]:Hu-w B n% l MN-j)N' B \ r8 . 

where N A is the number of A-individuals in community / at state j, N l B = iV — N l A is the number of B-individuals 
in community I at state j. Furthermore, denote l' xx the total number of interactions that an X-individual interacts 
with other X-individuals, and l' XY (X + Y) is the total number of interactions that an X-individual interacts with Y- 
individuals, where X, Y e {A, B}. Symmetrical ly, we have the following equations (l' AA N B )o = (I l BB N l A )o, (IabN b )o = 



(I l BA N l A )o at neutrality (INathanson et al. ■ l2009h ." The notation (-)o denotes the quantity averaged over all states of the 



stochastic process under neutral drift, w = 0. Thus, Eq. (8) can be simplified as 

<4X>o (l' AB N' B )o 
(«>o <I' BB N> 

ct' 1 



1 



(9) 



where cr' = (I AA N B )o/ (I AB N B )o. On the other hand, the newborn reproduced in community I migrates to community 
m with probability Ai m . Suppose that in community m there are h A-individuals and N - h /J-individuals, where /V is 
the size of community m. Similarly, we can get the effective sojourn time Ty, * 2(N—h)/N. And the effective number 
of encounters can be written as 

' cr m 1 
1 cr'" 



(10) 



where cr'" - </^^)o/</^') . 

Since we only discuss the number of interactions along a path that the mutant invades the whole population, the 
total number of interactions is a linear composition of the number of interactions in every community. So the total 
effective number of interactions can be finally written as 

I I) 

where cr = c/cr 1 + Y^mesi, c mO~ m - Here, the coefficient c„(n € / U Q;), reflects the strength of effect from effective number 
of interactions in community n to total effective number of interactions. 

Eq. (11) suggests that both types of individuals with the same strategy interact cr times as often as they interact with 
the other individuals. Therefore the average payoffs of A and B individuals along an invasion-path in a community- 
structured population are crR + S and T + crP, respectively. From this result we can get the cr law under the weak 
selection in a community-structured population, i.e., the condition that strategy A is more abundant than strategy B in 
the stationary distribution is <tR + S > T + crP. 
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3. cr law in the case of non-uniform interaction 



The <x law provide s a fundamental crite rion to specify which strategy is more abundant in a structured population. 
But, what's the <x? In iNowak et al. I J20 1 Oh . the authors said that <x is the 'structure coefficient', and it reflects how 



the structured population influences the evolutionary dynamics. In this section, we point out that cr also features the 
reaction rate between two individuals with the same strategy. 

To verify this point, w e will use the evolutionary game theory with non-uniform interaction rates discussed in 
Taylor and Nowak d2006h . They assumed that the probability of interaction between two individuals is dependent of 



their strategies, and described the interaction as 

A+A-^AA 
A + B^AB 
B + B-^BB 

Let us consider a simplified version of the prisoner's dilemma game (PDG) with the payoff matrix as below 

C D 

C b-c -c (12) 
D b 

If we discuss the game (12) in a population without community-structure, i.e. M = 1, strategy D dominates strategy 
C with uniform interaction rates r\ — ri — r$. Eventually, the entire population consists of defectors. However, if 
individuals only interact with opponents of the same strategy, cooperators cannot be exploited by defectors. In this 
case, when ri — and r\ , rj, > 0, cooperation is the dominant strategy due to b - c > 0. Therefore, r2 + means that 
cooperators and defectors do interact. 

Without loss of generality, we assume that r\ — rj, — r > 0, r2 = 1 in a population with community-structure. Let 
x 1 and y be the frequencies of individuals adopting strategy C and D in community I, respectively, and xf + y l = 1 . 
The fitness of individuals are determined by the average payoffs over a large number of interactions. Therefore, the 
fitness of A and B individuals in community I are 

f c = [r(b - c)x l - cy']/(rx l + y l ), f D = bx l j{x l + ry\ (13) 

In our model, an individual is randomly selected at each time step as a new vacancy, which is replaced either with 
the offspring of an individual from the same community or with an individual migrated from other communities. In 
community I, the probability of an X-individual (X = C, or D) filling a new vacancy due to local reproduction is 
proportional to the product of the number of X-individuals and their fitness, i.e., N' x f' x . And, the probability of an 
X-individual filling in a new vacancy due to global migration is proportional to the product of the averaged number of 
X-individuals and the migration rate A/, i.e., A/(Nx)- Here, we suppose A[ m — Ai = (1 - Au)/Nn n m e Q./, where Afo, is 
the number of set O;. Thus, we obtain the transition probabilities as 

T c^d = ( N 'cW ■ (N' D f' D + MN D ))/[£c,D(KfD + MN D ))] 

= (N' c /N) ■ if lif + A,) ■ (N l D fj,)/(Nf l ) + A,/(f' + A,) ■ (N D )/N] 

t 'd^c = W'cW ■ ( N 'cfc + MNc))/[ZcAN'rf[ + MN C ))] 

—i —i —[ —i (14) 

= (N' D /N) ■ If /(/ + Ad ■ {N l c f' c )l(Nf ) + A,/(f + A,) ■ (N c )/N] 

where /' = f c ■ (N l c /N) + f D ■ (N' D /N), and (Nx)-Tjfli N x /M. From Eq. (14), we know that after a vacancy appears, 

either local reproduction occurs with probability / /(/ + Ai), or global migration occurs with probability Ail {J + A[). 

For sufficiently large but finite populations, Traulsen et al. have shown that the stochastic process can be well 
approximated by a set of stochastic differential equati ons combining deterministic dynamics and diffusion referred to 



as Langevin dynamics dTraulsen et all 12005b, 2006c). Hence from Eq. (14), we may derive the Langevin equation to 
describe the evolutionary dynamics as 

x 1 = a l (x l ) + b l (x l )£ (15) 
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where a = 1 D ^ c - T C ^ D , b is the effective terms, and £ is the uncorrelated Gaussian noise. As N — » oo, the diffusion 
term tends to zero and a deterministic equation is obtained 

x 1 = a'(x') = (iV^/AO- [/'/(/' + A,) ■ (N' c fp/(Nf') + A,/(f' + A,) ■ (N c )/N] 
- (N' c /N) ■ [f'/(f' + A,) ■ (N'^/iNf') + A,/(f' + A,) ■ (N D )/N) 

Therefore, the equilibria of Eq. (16) which satisfy T' D ^ C = T l c ^ D are 

[-(a - 2/3)(x') 2 + (or - 2/3)x l + 0]/[-(r - l) 2 (x') 2 + (r - 1)V + r] = y/N (17) 

where a = r{b - c) - (b + c), /3 = -rc, and y = 4(N l c - N C )MA, with N c = E£i A^- 

Denote ANc = N l c - Nc- When ANc = 0, i.e., each community has the same numbers of C-individuals, Eq. (17) 
can be simplified as 

-(a - 2/3)(x 1 ) 2 + (a - 2/3)x' + = (18) 
When a > -2(3, i.e., r > (b + c)/(b- c), there are two interior equilibria. Specifically, the stable interior equilibrium is 

x' s * = 1/2+ V« 2 -4{3 2 /(2a-4/3) 

and the unstable interior equilibrium is 

x 1 * = 1/2- ^a 2 -4{3 2 /(2a-4p) 

As r increases, the two interior equilibria move symmetrically away from the bifurcation point x 1 * = 1 /2, the interior 
stable equilibrium moves toward 1, and the unstable equilibrium moves toward 0. Therefore, the proportion of cooper- 
ators tends to increase monotonically with r increases, and (b + c)/{b- c) is the critical value which specifies whether 
cooperators are more abundant than defectors or not. That's to say, cooperators are more abundant than defectors 
when 

r(b-c)-c>b (19) 

This condition coincides with Eq. (2), for R = b - c,S = -c, T = b, P = 0, and the interaction rate r plays the same 
role as <x in section 2. 

Since the frequencies of individuals adopting strategy C in each community become stable at the steady state of 
the evolutionary dynamics, ANc — * 0. Hence, when ANc + 0, we write ANc = ANco ■ s f° r some fixed AA^co > 0, 
and Eq. (17) can be simplified as 

[y(r - l) 2 - N(a - 2/3)](x 1 ) 2 - [y(r - l) 2 - N(a - 2p)\x l + (N/3 -yr) = (20) 

When Na + 2N/3 > y(r + \) 2 /N, i.e., r > (b + c + y/N)/(b - c — y/N), we obtain the stable interior equilibrium 

x'; = 1/2 + 1/2 Vfr(r + l) 2 - N(a + 2/3)]/[y(r - l) 2 - N(a - 2/3)) 
and the unstable interior equilibrium is 

x': = 1/2 - 1/2 VW + I) 2 - N(a + Wfm - D 2 - N{a - 2/3)] 

This conclusion is similar to the case of ANc = 0, that is, the two interior equilibria move symmetrically away from 
the bifurcation point x'* = 1/2 and the proportion of cooperators tends to increase monotonically with r increases. 
Thus, (b + c + y/N)/(b - c - y/N) is the critical value which specifies whether cooperators are more abundant than 
defectors or not. With simplification, we have the condition that cooperators are more abundant than defectors 

r(b - c) - c> b + (y/N)(r + 1) (21) 

When r > 1, a small s > leads to y — > 0. Eq. (21) is discriminated from Eq. (19) only with the infinitesimal term, 
and the second term in the r.h.s of Eq. (21) is negligible in the meaning of mathematic. Therefore, Eq. (21) also 
coincides with Eq. (2), for R = b - c,S = -c, T = b, P = 0. 

To sum up, Eq. (19) and Eq. (21) reflect that the interaction rate r plays the similar role as <x in section 2, and we 
verify that <x features the reaction rate between two individuals with the same strategy. 
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4. Conclusion and Discussion 



In this paper, we have proposed a model of games on a community-structured population to understand the cr law 
with the Moran process and the ergodic theory. By calculating the average payoffs of A and B individuals with the 
effective sojourn time method, we find that, <x features not only the structured populations characteristics, but also 
the reaction rate between individuals. Parameter <x, reflects that individuals using the same strategy may interact with 
themselves cr times than the interactions with individuals of the other strategy. That's to say, an interaction between 
two individuals are not uniform with the verification in a simplified PDG. 

In practice, an overlap of communities usually exists in social networks for example, yet the case with overlap 
doesn't be discussed in this paper. Here we simplify an overlapped community-structured population to a none-overlap 
community-structured population. For a simple example without loss of generality, suppose that a game is played in a 
finite population with the fixed size H distributed over 2 communities, which have one common individual, as shown 
in Fig. 3(a). In this case, the overlap individual can be regarded as two parts, itself and a virtual reproduction which 
exist in the two corresponding separated communities, respectively. Therefore, the overlapped community-structured 
population with size H becomes a none-overlap one with size H + 1 (see Fig. 3(b)), which is discussed in our paper. 
Hence, we provide a simplified yet alternative approach to study the evolutionary games in a community-structured 
population with overlap. 
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Figure 1. 

An illustrative community-structured population with strategies A and B. At each time step, a single individual from 
the entire population is selected proportional to its fitness for reproduction, and the offspring either stays in this 
community or migrates to another community. There exists a graph connecting all the communities in the structured 
population. The migration occurs between two communities if there is an edge connecting them both. 
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Figure 2. 

An illustrative fixation process of a single mutant in the whole community-structured population, (a) A single mutant 
is produced in community 1 (white node); (b) This mutant successfully takes over community 1 with probability An, 
and then migrates to community 2 to replace a randomly chosen individual with probability An', (c) The mutant takes 
over community 2 with probability A22, and then migrates to community 3 to replace a randomly chosen individual 
with probability A23; (d) The mutant takes over community 4 with probability A44, and then migrates to community 5 
to replace a randomly chosen individual with probability A45. 
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(a) (b) 

Figure 3. 

(a) An overlapped community-structured population with one common individual; (b) A population with two none- 
common communities. 
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